Gesture Synthesis from Sign Language Notation Using Mpeg-4 Humanoid Animation Parameters and Inverse Kinematics

This paper presents a novel approach for generating VRML animation sequences from Sign Language notation, based on MPEG-4 Face and Body Animation. Sign Language notation, in the well-known SignWriting system, is provided as input and is initially converted to SWML (Sign Writing Markup Language), an XML-based format that has recently been developed for the storage, indexing and processing of Sign Writing notation. Each basic sign, namely signbox, is then converted to a sequence of Body Animation Parameters (BAPs) of the MPEG-4 standard, corresponding to the represented gesture. Inverse Kinematics are also employed for synthesizing complex animation sequences (e.g. contacts). In addition, if a sign contains facial expressions, these are converted to a sequence of MPEG-4 Facial Animation Parameters (FAPs), while exact synchronization between facial and body movements is guaranteed. These sequences, which can also be coded and/or reproduced by MPEG-4 BAP and FAP players, are then used to animate H-anim compliant VRML avatars, reproducing the exact gestures represented in the sign language notation. Envisaged applications include interactive information systems for the persons with hearing disabilities (Web, Email, info–kiosks) and automatic translation of written texts to sign language (e.g. for TV newscasts).